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THE PROBLEM 

The study focused upon the relative efficacy of delayed vs immediate 
feedback of knowledge, i.e., information about the correctness of response. 
Despite widespread belief in the superiority of immediate feedback, support- 
ing evidence based upon the use of typical academic verbal material is ex- 
tremely scant. Using verbal material, this study explored the effect of 
the temporal interval of feedback as it interacted with two other variables: 

1. Method of presenting learning material: inductive or deductive. 

2. Activity of the learner during the delay interval: activity rele- 

vant to the material or irrelevant activity. 

Inasmuch as the learner's receiving immediate feedback were allowed no 
delay interval, the activity of such groups was varied a ft er the feedback. 



OBJECTIVES 

The major objective was to compare the effect of immediate vs delayed 
feedback of knowledge upon the learning of principles, using principles 
typical of the content of lower division college instruction. Secondary 
objectives were to explore the interaction of the major variable, indicated 

above, with two additional variables: 

1. The method of learning a principle: deductive vs inductive. 

2. The presence of non- informative material supporting relevant 
symbolic activity vs the absence of such material. 



HYPOTHESES 



1. Delayed feedback will result in superior verbal learning. 

2. The presence of non-informative symbolic supporting will result in 

, , i to the absence of such material. 

superior veroax xcaiui»»6 -- 

3. The interaction between temporal interval of feedback and supporting 
material will be significant. Such significance will result from 
the positive learning increment due to both delayed feedback and 

induced syinbolic setivity* 

4. The hypotheses indicated above (1,2, and 3) will be supported by 
both inductive and deductive methods of presentation. 



RELATED RESEARCH 

The view that learning varies inversely with the amount of time intervening 
between a response and reinforcement is incorporated in several prominent 
theoretical approaches to learning (Hull. 1952; Spence. 1956; Miller, 1950; 
Skinner, 1938. 1957). Furthermore, this notion of the superiority of immediate 
reinforcement has been included in almost all attempts to apply principles of 
learning to education practices. Current textbooks in educational psychology 
often cite this as a fundamental principle of learning. Klausmeier (1960) 

Reinforcement immediately following the correct response usually 
fflrilitates learning more than does a delayed reward or r.in 
f^^cefe^^! The teacher must know what the co-ct 

SlUy f^ tL improvement of all human abilities seems to lie 
at this point... 

Blair, Jones, and Sin^,son (1962) suggest. "From what is presently known, feedback 
is more effective if imnediate and specific." Bigge and Hunt (1962) state, 

"The key to effective teaching of thinking, as any other behavior, is immediate 
feedback." Similar claims for the superiority of immediate reinforcement in 
classroom learning can be found in almost any modern textbook of educational 



psychology. 
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The booming expansion of teaching machines has further emphasized this 
principle. All current programs for teaching machines appear to be based on 
the notion that immediate reinforcement is the most effective condition for 
learning. Holland (I960), co-worker of Skinner and a key figure in the develop- 
ment of teaching machine programs, suggests that of several well-known learning 
principles to be applied to teaching machine development, ’’immediate reinforce- 
ment for correct answer is a must...” Similar authoritative assertions, 
proclaiming the necessity of immediate reinforcement, are found throughout the 
literature pertaining to teaching machines and programming. In teaching machine 
programs in programmed tests as well, the reinforcement consists of either 
informing the student of the correct answer or telling him whether his answer 
was right or wrong. 

Although this new medium focuses attention upon the importance of immediate 
reinforcement, its application is not limited to programmed learning. Broadly 
conceived, reinforcement is any consequence following a response which results 
in increased likelihood of that response in the future. Thus, telling a student 
that his answer is right or wrong, handing back tests, assigning grades, the 
teacher's approval of any student behavior are all examples of reinforcement. 

Much of the teacher’s behavior may be considered as the application of reinforce- 
ment. Consequently, valid principles of reinforcement have broad applicability 
to educational practice. Therefore, it is crucial that any asserted principles 
be validated upon relevant human learning problems. If the principle of the 
superiority of immediate reinforcement is tenuously founded or even erroneous 
when applied to typical classroom learning, educational progress may be 
seriously retarded. 



Empirical evidence for the princple of immediate reinforcement has been 
derived from three types of studies: 

A oni mn 1 ^aYn^>T•i ment?; 

< V • '-'-'■X' 

B. studies of human motor skills 

C. experiments using typical classroom material 

Numerous animal studies supporting the immediate principle can be found 
in any experimental learning text. Samples might include: Wood (1933), Perin 

(1943), Grice (1948), Carolton (1954 and Myles (1958). However, a slender, but 
persistent, current of conflicting evidence appears even among animal learning 

experiments . 

Such early studies as Warden and Haas (1927) indicating no inefficiency 
from delay were later explained by positioning a secondary reinforcing effect 
derived from cues, e.g., the goal box, previously associated with the reinforcing 
food. The power of derived or secondary reinforcers to bridge the delay gap has 
been heavily relied trpon by major learning theorists: Hull, Spence, Mourer, and 

Skinner. Such reasoning would imply taat highly distinctive cues during the 
delay interval might remove any deterious effect. Results consonant with this 

were found by Renner (1963) . 

The function of the delay interval as a source or catalyst of motivation 
is suggested theoretically by both Mourer (1960) and Spence (1957) . Empirical 
support for such an effect is found in studies by Raymond (1954) and Bebeler 
et al (1957). In these studies speed of locomotion was the dependent measure. 
The rodent subjects displayed faster running under delayed conditions. 

The effect of delay upon animal performance needs to be examined in light 
of the subject’s previous experience with temporal dimensions. Studies in 
which training trials utilyzed increasing or variable delays Dews (1960), 

Logan (1960) support that both accuracy of discrimination and speed of response 
can be facilitated by controlled use of delayed feedback. 



Subject to the above considerations, rat experiments generally show that 
speed and latency o£ response are adversely affected by delay of reinforcement. 
However, if other measures of learning are used, the effect of delay is by no 
means clear-cut. Evidence o£ learning interference from delay was found by 
Perin (1943b), Grice (1948) and Lawrence and Hommel (1961). "However no 
effect of temporal delay upon the number of trials to acquisition was found 

by Renner (1965), Roberts (1930) and Wolfe (1934). 

Evidence from human motor ^kill tasks casts further doubt on the 
superiority of immediate reinforcement.' Several studies (Lorge and Thorndike, 
1935; Saltzman, Kanfer, and Creenspoon, 1955; Noble and Alcock, 1958; Denny, 
et al, 1960) found no difference between immediate and delayed feedback upon 
the acquisition of such responses as drawing lines of a specified length, 
moving a bar, pulling a lever, etc. Bilodeau and Bilodeau (1958) in a series 
of five studies found no interference on motor skill from delay, and in one case 
delay resulted in improved learning. In their' review of the effect of delay 
of reinforcement upon motor skills learning Bilodeau and Bilodeau (1961) 
conclude, "Immediate knowledge of results is immaterial to learning so long 
as the time intervals are free from specially interpolated responses 

Until recently, few experiments had investigated the effect of immediate 
vs delayed reinforcement using typical classroom material. The earlier studies 
(Angell, 1949; Little, 1934; and Pressey, 1950) indicated superior learning 
under conditions of immediate reinforcement. However, in each study the method 
of administering reinforcement differed between immediate and delayed groups. 
Thus, in Angell’s oft -cited comparison of freshman chemistry students, the 
immediate reinforcement group used a punch board device to receive feedback 
while the delayed reinforcement group' used conventional answer sheets. No 
resulting differences can be clearly attributed to temporal differences in 
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feedback. Differences in mode of administration, novelty effects, and various 

interactions present possible sources of variance. 

Renner’s (1964) review of literature on delay of reinforcement does not 
O <=ini,le atudv in which verbal stimuli were used. Since the review, 

W ww — ““O — - ' ^ 

several studies employing verbal material have raised doubt as to the superiority 
of immediate feedback. Jones and Bourne (1964) report several studies explor- 
ing the effect of a short unfilled delay interval. No difference was found in 
performance on a verbal maze using 0-sec. vs 6-sec. delay. On a paired 
associate task performance improved with increases in both the delay interval 
(0, 3, and 6 sec.) and in the post^delay interval. Such effects were additive. 

A series of studies by Brackbill and associates (Brackbill, Branos, and 
Stair, 1962; Brackbill, Isaacs, and Snelkinson, 1962; Brackbill and Kappy, 

1962; Brackbill, Bohlitt, Darlin, and Wagner, 1963; Brackbill, 1964; and 
Brackbill, Wagner and Welson, 1964) question the validity of the principle 
that delayed reinforcement has a detrimental effect on human learning. Further 
studies (Bilodeau and Bilodeau, 1958; Boulter, 1964>; Brackbill and Kappy, 

1962; Ryan and Bilodeau, 1962; Crawford and Sturgis, 1964) on human motor and 
verbal learning problems show no difference in- learning due to the time of 

reinforcement « 

The majority of the investigations- on delayed reinforcement measure 
acquisition as evidence of learning-. The number of errors or the number of 
trials required to attain a prescribed criterion is the measure. However, 
Brackbill, Wagner and Wilson (1964) stress the fact that for practical purposes, 
it would be more useful to investigate the effect of delayed -reinforcement on 
retention. The authors (1964) feel that more emphasis should be placed on 
retention inr ead of on the sole process of acquisition. 
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Brackbill, Wagner and Wilson (1964) required third grade children to learn 
eighteen English words and their French equivalents, recording ' the number of 
errors and trials to reach a criterion. The difference between'the immediate 
and ten*'Second delayed knowledge of results (KR) treatments were insignificant. 
However, when the same measures were taken again seven days later to determine 
how well the subjects were able to relearn the materials, the 'delayed KR 
group did significantly better. Retention, the number of errors in y.elearning 
or the number of trials to relearn, has been measured by this study and by 
Brackbill, Bravos and Starr, 1962; Brackbill and Kappy, 1962; Brackbill, Isaacs 
and Smelkinson, 1962; Brackbill, Warier and Wilson, 196^; Lavery and Suddon, 
1962; and the results indicate that delayed KR improves retention while 
immediate KR impairs retention. 

However, it is difficult to assess whether the difference in retention 
is due to the delayed KR or to the fact that during acquisition, the subjects 
may have been exposed to the materials to be learned an unequal magnitude of 
time. The subjects receiving delayed KR in the study by Brackbill, Bravos 
and Starr (1962) and Brackbill and Kappy (1962) required more trials to 
acquire the material to be learned and consequently were presented with the 
material to be learned a greater magnitude of time than were the subjects who 
I'^arned more rapidly. After the material has been presented' equally to all 
subjects, a measure of acquisition may be taken, however, it' is difficult to 
conceive a measure of learning as reflecting only acquisition or only reten- 
tion, since these two constructs are always confounded in any actual behavior. 

Although the majority of the studies, typically using rodent Ss, reviewed 
by Renner (1964) indicate that learning efficiency decreases with increase in 
feedback delay; however, when an attempt is made to control for any mediating 
variables during the delay interval, the results are not as conclusive. Grice 




(1948), Perkins (1947) and Marker (1956) demonstrate the ineffectiveness of 
delayed reinforcement due to the mediating of secondary reinforcing agents and 
Renner (1968) concludes from his study, "...that the temporal gradient of 
reinforcement is a function of drive level and availability of cues." Carolton 
(Spence, 1960) devised a confinement segment in his apparatus -which, "...would 
discourage turning away from the food-.-cup during the delay period and thus 
increase the likelihood of maintaining orientation toward it;" This increased 
control of the rat's activity during the delay interval was shown to be a 
variable which significantly facilitated learning over the rats which were not 

confined. 

The bulk of the studies (Alexander,' 1951; Bilodeau and Bilodeau, 1958; 
Bourne and Bunderson, 1963; Brackbill, Isaacs and Smelkinson, 1962; Brackbill 
and Kappy, 1962; Brackbill, Wagner and Wilson, 1964; Greenspoon and Foreman, 
1956; Landsman and Turkewitz, 1962; Noble and Alcock, 1958; Ryan and Bilodeau, 
1962; Saltzman, 1951; Saltzman, Kanfer and Greenspoon, 1955) on delayed rein- 
forcement appear to overlook the time intervar between response and reinforce- 
ment for any possible effective variable other than the mere passage of time. 
The studies do not attempt to control for the subjects »■ activities or stimuli 
which may possibly be interf erring with or facilitating learning. The 
assumptive framework of such studies appears to regard this time as a temporal 
vacuum in which “nothing" impinges upon the subject. Obviously, such a state 
is not experimentally producible at the present and the variables that do 
occur during this interval of time must be controlled. 

The importance of the delay interval' for variables other than solely the 
passage of time was also indicated in Jones and Bournes (1964). Results showed 
that delay was detrimental only as a function of successive items presented 
prior to KR. Ross, Hitherington and Wray (1965) demonstrated a poorer 
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performance of children in a size discrimination problem due to the continual 
presence of the stimulus during the delay interval. The authors (1965) 
attributed the effect to competing responses made during the delay interval. 

Hockman and Lipsitt (1961) - . . . supposea max enectb ux 

dependent upon the effective distinctiveness of the stimuli to be discriminated 
or the difficulty of the task," and in the experiment (1961), merely decreasing 
the number of stimuli to be differentiated likewise decreased generalizatio. 
among them and thus enhanced the learning rate. Similarly, Rieber (1961) 
hypothesized that the delay of reward in children is to facilitate the associa- 
tion of competing responses with the stimuli which elicit the conditioned 
response. Rieber (1961) concludes from the study that "Hence, it would be 
expected that interference with the conditioned response would be an increas- 
ing function of the similarity between the cues present during the delay 
period and those which elicit the conditioned responses." The conclusion 
of Noble and A1 cock (1958) that "Whether reward or information is withheld 
seems to be of less consequence than what the subject does during the time 
interval between response and after effect." seems increasingly apt. 

Saltzman (1951), in a study described earlier, attributed the poorer 
performance of the delay group to their interf erring activity during the pre- 
reinforcement interval. The activity was rehearsal of :the presented stimulus 
and since rehearsal was occurring prior to knowledge of the' correctness of 
response, the incorrect response was reinforced as well as the 'correct one 
and interferred with acquisition. Brackbill, Bravos and Starr (1962) assume 
that the rehearsal activity is a major variable in learning; However, it is 
their (1962) contention that these covert responses are being strengthened, 
due to their being followed by reinforcement. Immediate reinforcement is 

reinforcement precedes rehearsal. 



not as facilitating since 
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The results from an investigation by Sturgin and Crawford^ (1964) showed 
no differential effect in verbal learning due to the time of reinforcement and, 
in fact, one experiment revealed delay to be superior to immediate reinforcement. 
Sturgis and Crawford conducted four experiments which compared the effectiveness 
of immediate and delayed reinforcement upon different types of learning material 
including nonsense syllables, factual material, and inductive'' generalizations. 
Reinforcement was presented in two forms r complete knowledge of results, and 
cues directing the subjects toward the correct answer. 

In each experiment subjects were assigned to one of three groups: Immediate 

Reinforcement Group, in which knowledge of results was given immediately after 
answering each question; Delayed Reinforcement Croups in which knowledge of 
results was delayed for 24 hours; Control Group, in which no 'knowledge of results 
was given. Subjects were presented with learning material .consisting of 
multiple-choice questions and their answers recorded immediately after seeing 
each question. They then received either immediate, delayed, or no reinforce- 
ment. All subjects were tested on the original learning material one week later. 
In each of the experiments the number of presentations of the material, the 
type of presentation of the material, the' length of exposure of the material 
and the form of reinforcement was equated for all three. groups. 

The results of the four experiments indicated that' in no case was 
immediate reinforcement superior to delayed reinforcement. When reinforcement 
was in the form of complete knowledge of results and when the type of material 
was either factual or inductive, the delayed reinforcement was more effective. 
Only when the material consisted of nonsense syllables or when reinforcement 
was given in the form of cues did immediate reinforcement equal that of delay 



in effectiveness. 
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The discrepancy between the findings of this study and those of the 
majority of previously reported studies in this area could be due to a number 
of different factors. Two plausible factors are the type of response and the 
specific function of reinforcement. The study dealt. with verbal behavior in 
human subjects. Both the stimulus and the response were in the form of 
symbols, of which the subject has a large repertoire. Conditions were conducive 
for the subject to indulge in mediated activity, in the . form- of symbolic 
exploration of the material, during the interval between the' response on 
record and reinforcement. Simple bar-pressing responses, motor skills on 
specialized apparatus, use of subjects from a lower species -.with a less com- 
plex nervous system, or subjects with scant symbolic repertoire would mitigate 
against any such mediating activity between the response and' reinforcement, and 
against any such advantage caused by the delay of reinforcement. Thus, the 
results of preceding studies may favor immediacy because the experimental 
conditions were not conducive to mediated" symbolic activity 

This imp3ies that varying the probability of medi at ed^• symbolic activity 
would result in varying the effective temporal interval of -reinforcement . 
Conditions should increase the efficacy of delayed reinforcement : ■ Conditions 
tending to diminish symbolic activity should favor ..immediate reinforcement. 

The above rationale which is consistent with the. previous findings of 
the present authors suggests the need for further exploration of the relative 
effectiveness of immediate versus delayed reinforcement under varying academic 
learning conditions. Much typical classroom learning; appears to be primarily 
concerned with the learning of concepts, i.e., classes of .events, or of 
principles, i.e., relationships between concepts. Concepts and principles may 
be learned by either an inductive or deductive method. The present study was 
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in part an attempt to investigate the effects of simple intervals of reinforce- 
ment upon the learning of concepts and principles by both inductive and deductive 
techniques <, 



rurtiiex'inui c, me 



^ oVirkiTts Tw-kin+c tr» thp coTicliision that 

A CIUJLV^UCIX^^ Vi O X -t V* ^ ^ 



any conditions furthering relevant symbolic exploration of learning material 
will promote learning o Thus, interposing related stimulus material which 
although generally relevant in content does not :ontain the answer to the 
original question, should promote such symbolic activity and thus result in 
increased learning » 

One limitation of the present authors* previous study was that learning 
was measured by recognition of the identical learning material « Probably a 
more useful measure and one analogous to the goals of classroom learning is a 



recall measure utilizing generalizations of the learned material to new 



instances. 

Thus, the present series of studies was designed to explore the following 
variables and their interactions: a) temporal interval of reinforcement; 

b) inductive and deductive methods of learning concepts and principles, c) inter- 
vening stimulus material designed to promote symbolic activity but of a non- 
informative character. The measure of learning was generalization to new 
examples using a method of recall. 



PROCEDURES 

Design 

A 2x2x3 Factorial design was utilized to explore the effects and inter- 
actions between the three variables of concern: 

Variable A: The temporal interval between response and reinforcement. 



15 



Variable B: The presence of non- informative material designed to promote 

relevant symbolic activity during the pre-reinforcement 

XAI. A W v^-%. • 

Variable C: The method of learning— inductive or deductive. 

Levels examined within each variable were: 

1. For variable A, two levels: immediate, and delayed reinforcement. 

2. For variable B, three levels: the presence of material which was 
non- informative, but designed to promote relevant symbolic material; 
the presence of material which- was non- informative -and not designed 
to promote relevant symbolic activity; and the absence of any such 

material. 

3. For variable C, two levels: an inductive presentation, and a deductive 

presentation. 

The design thus required twelve t’.eatmenf groups, two temporal levels, two mode 
of inference levels, and three interpolated material levels. An outline of the 

design is portrayed in Table I. 

DESIGN OUTLINE 
Table I 



®1 ®2 

Cl C2 Cl C2 Cl C2 


^2 

h ^2 

Cl C2 Cj C2 C2 


Ai Bi Cl 


A2 Bi Cl 


Ai Bi C2 


^2 ^1 S 


Ai B2 Cl 


A2 B2 


Ai B2 C2 


A2 B2 C2 


Ai B3 Cl 


^2 ^3 ^1 



Ai B3 C2 



A2 B3 C2 
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Subjects 

The subjects were two hundred and seventy-seven (277) lower division 
college students. All subjects were selected from one large (enrollment 
approximately 400) introductory psychology class. Students in this class 
had the option of participating in several experiments or writing a term 
paper. In this sense, all were volunteers. Students not participating in 
the experiment proper served with members of ot er preceding classes, as 

subjects in' a series of pilot studies. 

Employing a numerical code to identify subjects, a giaduate assistant 

randomly assigned subjects to groups and groups to treatments. 



Apparatus 

An ”800" Carousel slide projector was used to present the material. 

The learning material, including questions, answers, the similar and 
dissimilar material presented during the pre or post reinforcement intervals, 
etc., was photographed and made into 2" by 2" slides. Mimeographed question 

sheets were used in the test session. 

The general experimental procedure was essentially the same for each 
group. The subjects were presented, by means of a slide projector, with the 
initial learning material in the form of multiple choice questions. The 
subjects made a response by marking the answer they thought correct and then 
received knowledge of results (KR) immediately or five minutes later. Material 
to elicit relevant and irrelevant activity was shown on the screen during the 
pre-reinforcement interval for delay groups or post-reinforcement interval 
for immediate KR groups. The subjects were tested twenty- four hours later. 

The experiment was conducted in two sessions during the regular class 
period. Students were not informed of the retest. They were exposed to the 
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treatment conditions in small groups, seven to ten students. The procedure 
for the two sessions as follows: 

First Session^. A booklet of six answer sheets was passed out to the 



subjects as they entered the ruum and took their 



S63.US « 
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the group they were participating in a learning study concerned with the 
effectiveness of presenting materials in differnt ways and that their coopera- 
tion was essential to the outcome of the study. The following instructions 



were then given: 

"You will be shown questions, onthe screen, one at a time. 

IVhile the question is exposed, think about the question and 
answers and when I give the work, you will have 15 seconds to 
fill in the correct answer. Do not answer the questions until 
I give the work, but you must fill in an answer<> After you have 
filled in the answer, you will be instructed to tear off the 
sheet and turn it over. Attempt to learn the correct 'answer." 

The subjects were then told to remain seated and refrain from talking during 

the experimental session. 

All subjects were presented a multiple choice question. For the inductive 
groups the stem of the question consisted of an example of the principle. The 
alternatives consisted of the correct principle exemplified in the stem, and 
three incorrect principles. For the deductive groups the stem consisted of 
a statement of the principles; the' alternatives were three non-exemplars and 
one correct example. After one minute, each group was instructed to mark their 
selected answer, tear off the answer sheet and turn it over. 

1. The delayed reinforcement groups given irrelevant .material during the 
interval were then presented with fifteen German prepositions and 
their English equivalents on the screen with the instructions, "Attempt 
to memorize these German words;" 

2. The delayed reinforcement groups given relevant material during the 
interval were presented with material similar to the concept to be 



n 
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learned and told "Here is some information relevant to the question, 
attempt to learn it." 

3o The delayed reinforcement groups given no material during the interval 
were presented a blank screen for five minutes and given nc instruc- 
tions . 

This sequence was replicated six times;- each presentation consisting of a 
differe.it principle to be learned. 

The procedures for the immediate reinforcement groups -were essentially 
the same, however, the supporting' material was presented during the post- 
reinforcement rather than the pre-reinforcement interval. 

The temporal sequence of question (Q) response (R) and knowledge of 

results (KR) was as follows: 

1. For the delayed feedback groups. 

It i5*t 5? 30" 10" 1’ 15" 



Ql 



KR. 



Q2 R2 



2. For the immediate feedback groups. 

1‘ 15" 30" 5* 10" 1’ 15" 



Ri 



KR, 



^2 ^2 



• • • 0 • • 



During the five-minute interval either - the material designed to stimulate rele- 
vant symbolic activity, the material not so designed, or a blank screen was 

presented. 

At the end of the session, each group was gi en instructions as follows: 

’i.a/ik you very much for your cooperation. It is very important 
that you do not discuss this experiment with anybody. We will 
be glad to discuss this experiment with you anytime after Monday. 

Thank you again for your cooperation, you may now leave." 
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Second Session . The subjects were retested” twenty-four hours later, in 
their classes. The retest, consisting of two parts, was: 

1„ A test sheet with the general title of the six principles and 
instructions for the subjects to elaborate on or describe the 
principle more specifically <> 

2o A test sheet with six multiple choice questions; each question 
consisted of the title of a general principle and four possible 
examples of the principle. 

Ten minutes was allowed for the completion of each part. 

Materials 

A review of standard published sources failed to disclose suitable learning 
material for this study. Consequently, materials were developed. Considerations 
guiding the development of the material to be learned .were: ' 

1. Subjects prior to the treatment would be unable to identify principles, 

or examples at greater than chance levels. 

2. Subjects could display some evidence of learning -after relatively 
brief treatments. However, learning would nof. occur' to such a 
degree that perfect performance would mask sources .of variance. 

3. Principles were similar to those found in lower division college 
course materials. 

4. A number of exemplars and non-exemplars of each principle could be 
identified. 

Pilot studies were devoted to such development. During Fall and Winter Quarters, 
over 400 subjects were involved in a series of pilot studies directed toward 
the development of appropriate materials and procedures. Students enrolled in 
the same course designation as that from which the treatment subjects were 
selected were employed in these pilot efforts. 
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The principles as selected are contained in Appendix A. 

Supporting material, for Variable B as described in the design of the 
experiment, was developed for each principle so that the supporting material 
referred to the same content area and instructed the subjects "to think auuui. 
relevant categories within that area, but did not offer. any information from 
which legitimate inferences identifying principles or examples- could be made. 
Relevant material consisted of brief paragraphs of two to four sentences 
concerning the content area from which the principles were selected. Only 
concepts consistent with those of the principle' in question* were included. 

Pilot studies indicated that subjects did not, either. logically or illogically, 
leap to correct inferences solely from the supporting. material. The develop- 
ment of material designed to engage the student in symbolic activity, but no^ 
principle relevant, led to the use of German words. Fifteen prepositions were 
finally selected. Subjects were instructed to learn the. German word and its 
English equivalent. Pilot Ss reported sustained involvement in this task 
and were unable to recall any process or content' that appeared related to the 

principles to be learned. 

The relevant supporting material'is contained in Appendix the 
irrelevant in Appendix C. 

ANALYSIS OF DATA AND FINDINGS 

The study was designed as a 2x2x3 factorial design*. Data were analyzed by 
a factorial analysis of variance using the hypergeometric correction due to 

somewhat unequal n’s in several sub groups; 

Two measures w'=*re taken: An essay recall of principles and a multiple 

choice matching of principle with a new* example; The subjects’ performance 
in the six essay- type questions were rated by judges according to specific 
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predetermined criteria (Appendix D) and scored on a zero, one-half, one, and 
two point scale. Three graduate student raters'were instructed" in the applica- 

^ ^ cx-r-A on/1 ■? V T)GT£oTTIl6d th.6 TBtlH2S» IlltGrX&tCr 

KJJL X ✓ X 



reliability exceeded o90. 

An analysis of variance of the rated scores was performed to compare main 

effects nteractions. No significant effects were. revealed in any of the 

comparisons. Thus, with respect to recall of- principles. none' of' the' variables 

simply or- in combination displayed any significant relationship to" the outcome. 

The second measure, the matching of principle to new example was similarly 

analyzed in a 2x2x3 factorial design. Factors and levels are .as ' follows: 

Factor A signifies the mode of presentation: = inductive; A^ = deductive. 

Factor B signifies the kind of material presented during the" delay interval for 

delayed group and after feedback for the immediate group,: The three levels 

represented within factor B are: = relevant material; - no material; 

B = German words. Factor C signifies the temporal measure; of feedback: 

3 

C = delayed; = immediate. Results on this analysis of variance are 
1 2 

summarized in Table II. 
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SUMMARY 


OF ANALYSIS OF 


VARIANCE 








Table II 






Source 


SS 


df 


MS 


F 


A 


28,55 


1 


28.55 


2.17 


B 


106.85 


2 


53.43 


4.07* 


C 


2.21 


1 


2.21 


— 


AxC 


122.62 


1 


122.62 


9.34** 


BxC 


50.79 


2 


25.39 


1.93 


AxB 


2,73 


2 


1.36 


— 


AxBxC 


41.13 


2 


20.56 


1.56 


Treatments 

Within 


354.88 

3477.92 


11 

265 


13.12 


■k 

•k* 


Total 


3832.80 


276 







Only one main effect was significant, B, the kind of interpolated material 
presented. Orthogonal comparisons between levels within factor B' show that the 
effect was due to the divergence of level from the other two levels. B^, the 
similar material, showed significantly- lower retention than either 'the irrelevant 
German words' B2 or the absence of any such material. There were no differences 
between groups presented German words and those presented a blank screen during 
such intervals. The orthogonal comparison within B are stmimarized in Table III. 
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SUMMARY OF ORTHOGONAL COMPARISONS 
WITHIN FACTOR B: SUPPORTING MATERIAL 

Table III 

n CO MC 

OUUX'CC OO ux 



vs B^ + B^ 91.4 

B^ vs B^ 17.74 



1 91.4 

1 17.74 



Within 



109.14 2 

3477.92 265 13.12 



t; 

A. 

6.97** 

1.35 



One interaction was significant; that between mode of presentation and 
temporal feedback. This A x C interaction shown- in' Table. II reveals that: 

1. Within the inductive treatment' the immediate feedback groups 
perform significantly better- than- the delayed feedback- groups . 

2. Within the deductive treatment*, the delayed feedback groups perform 
better than the immediate feedback- groups. ' 

Briefly, immediate KR appears superior for inductive learning, delayed 
KR appears superior for deductive learning-. 

SUMMARY OF ORTHOGONAL COMPARISONS 

WITHIN INTERACTION A x C: TEMPORAL INTERVAL AND METHOD OF LEARNING 

Table IV 

Source ss df MS F 

Aj^C2VS A-j^C2^+A2Cj^^*A2B2 113.3 

A2Cj^vs AjCj^+ A 2 C 2 31.76 

A 2 C 2 vs AjCj 11.29 

166.35 
(122.62) 

Within 3477.92 



1 113.3 8.64** 

1 31.76 2.42 

1 11.29 

3 

265 13.12 



Table IV indicates orthogonal comparisons within this interaction. The 
significant component compares the inductive^delay group against the -weighted 
sum of the inductive- immediate group, the deductive- del ay group, and the deducti^ 
immediate group. All other interactions and components display no significant 

differences . 

CONCLUSIONS AND IMPLICATIONS 

Four hypotheses were stated earlier in' this report. They were. 

1„ Delayed feedback will result in superior verbal learning. 

2, The presence of non- informative symbolic supporting' will result 
in superior verbal learning as compared to the absence of such 
material. 

3. The interaction between temporal interval' of feedback and supporting 
material will be significant. Such significance'.wiil result from the 
positive learning increment due to both delayed feedback and induced 

symbolic activity. 

4o The hypotheses indicated' above (1, 2, and 3) will be supported by 
both inductive and deductive methods of presentation. 

As stated, none of the hypotheses were supported. 

The major objective of the study was to compare' the- effect '.of- immediate — 

delayed KR using instructional material typical of college. courses. ^ overaU 

differential effects were found. This- however, masks- the .significant and pro- 
vocative interaction between delay of- KR and mode of' inference; That delayed 
KR would result in superior learning for' the deductive treatment groups, while 
immediate KR would result in more learning- for- the deductive treatment groups 
was not expected. A first-order conclusion' seems warranted. The universal 
superiority of immediate reinforcement did not appear. Over a major subset 
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of treatment conditions, six groups, there was not only no superiority but a 

significant inferiority in learning outcomes associated with' immediate KR. 

. < ^-r* o o ■f*V» twrfi^T'a 1 1 

Of course, the converse is aiso rrue, a gcucxaxx^<xuxv.x*- — 

superiority of delayed KR with academic material cannot .be supported. 

The peculiar interaction between modeof inference: and" timing of 
reinforcement suggests the flow of somewhat different processes "occurring 
between induction vs deduction so that the timing of KR, results: in a differential 
effect. One possibility is that the sort of principles. used' in these experiments 
were readily retained symbolically over the five-minute -delay interval. Experi- 
mental conditions apparently were conducive to frequent- rehearsal of the 
principle. Although a search for the correct examples might' have been unsuc- 
cessful, the continuous rehearsal of principle rendered the subsequent KR 
more effective. In the case of the inductive treatment .groups', only the example 
was available for rehearsal before KR. Thus, no enhancing effect was possible. 

The inductive groups, performing symbolic manipulations' from" the focus of the 
example had little basis to conduct' such reiterated processing. Principles 

are more useful tools to carry than examples. 

A puzzling result was not only the lack of positive effect from" the 
materials designed for relevant symbolic support but' their- apparent interfeg .n g _ 
effect. Learners with such intended support learned less than the other groups. 

The lack of any positive' increment could be due to the' characteristics 
of the learning setting. It was test-like. The college subjects:are notoriously 
test-motivated. and over the brief five-minute period'. no: symbolic support was 
needed. However, the apparent negative effect remains" difficult to explain. 
Observations of the subjects indicated that subjects: in' both the relevant material 
and the no material groups began looking around the room, scribbling on tbeir 
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answer sheets, twisting and squirming, etc» The material may have failed, 
despite supporting evidence from pilot subjects, to maintain the desired covert 
processes over the interval. If so, the material itself may here become a 
discriminative stimuli for behavior contradictory to the desired learning. 

The m.aj or educational implications of the study are -twofold: 

1 . The usual assumption of the efficiency of . immediate reinforcement 
is not valid with respect to academic ' instruction a . 

2 o Ttie apparent superiority of delayed KR with deductive learnin g and 
immediate . KR with inductive learning may be utilized across a wide 



range of instructional settings . 
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Reinforcement in the Form of Knowledge of 
Results for Each of the Six Responses 
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Reinforcement in the Form of Knowledge o£ Results 



Reinforcement 1 

The intensity of sensation is equal to the intensity of the physical stimulus 
squared (Multiplied by itself). 

Reinforcement 2 

Adolescents have trouble relating to father figures. 

Reinforcement 3 

men two mixable liquids which do not react chemically are placed in the same 
vessel, a slow mixing process occurs from the molecular motion and the liquid 
becomes uniform throughout. 



Reinforcement 4 

An argument from an accepted rule or principle to a special case, when the 
rule is not applicable to the special case. 

Reinforcement 5 

Learning a single concept is facilitated by all positive instances, but this 
interferes with later learning of more complex concepts. 

Reinforcement 6 

If you do something in a given situation, the next time you are in that situa 
tion, you will tend to do the same thing. 
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Relevant Material 
Material Relevant to Stimulus 1 

^o-j-ArtioTi. recognition, discrimination, and scaling, 
?;r;;;;’oiTseg»ent o£ experimental psychology called psy^o- 
physics. The name psychophysics derives from the classical 
the relation between the physical environment and the "iind. 
psychophisicists are not professionally ®°"®®™f 

isLe of the mind-body relation, but rather with the constraints that are 
placed ui<on the behavior of a person in his jud^ents, actions, and so on, 
by the sea of physical energies that surround him. 

Material Relevant to Stimulus 2 

rhildhood continues up to the time when the child can get on fairly well 
with his peers- the juvenile era begins when playmates are badly needed and 
arf in mosrw^ys, preferred to adults. The "eruption, due to maturation, of 
a need for an intimate relation with another person of comparable status 

Lrk: theVegiining of pre-adolescence a relatively b^^^rrsf ffo"a per- 
ends with puberty. Adolescence is marked by a shift of interest from a p 

son of one’s own sex to one of the opposite sex, and 

adult sexual activity. At adulthood one is able, or e as^important 

establish a love relationship in which the other person is almost as important 

as oneself. 

Material Relevant to Stimulus 3 

TViP first sten in applying the scientific method is to obtain some facts, by 
I£:eriafioraSd The next step is to classify and correlate the 

facts by general statements. If a general statement is simple i ^^ipotv 
IT^e Jled a law of nature. If it is more complex it is called a theory. 

Both laws of nature and theories are called principles. 

Material Relevant to Stimulus 4 

In logic an argiment is a group of two or more statements, °"® ”^^®^A?nt,ed 

affirmed on the basis of the other or others. The statement which is affirmed 
if called thf conclusion of the argument. The statement or statements which 
supply the reason or reasons for affirming the conclusions are called the 

premises of the argument. 

Material Relevant to Stimulus 5 

Concepts are condensations of past experience. They bring ® 

idL ^so to speak, what has been learned about pr(^erties of many different 

for example the concept tree. This concept is foreign to 

certain Australian tribes. The native speaks of particular objects, like the 
TarX t"e muigl and the gum, but he has no word to represent what is common 

to them. 
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Material Relevant to Stimulus 6 

An example of laboratory learning, on classical conditioning, is the conditioning 
of the eyeblink reflex in humans. If a person who is watching a dim light sees 



reiiex J-ii uumcwio^ xx ^ 

hriohter. he ordinarily does not blink his eyes in response 

irthirsti;ulur'"lf:'how;ver; he is hit in the eye by a vigorous puff of air 
does InZ ihe c;nditioning procedure consists in pairing these two stimuli, 
with the brightening of the light coming a fraction of a second before the p f 
of air Each time this sequence occurs, the subject blinks in response to the 
ah puh. Presently, however, he begins to blink as soon as the ight changes, 
Tefo^e the puff comL. Since the changing light now produces ^^Uuking response 
which it fonnerly did not produce, learning has taken place. In this setup the 
puff which already produced blinking, is called the unconditioned stimulus, and 
blinking to the puff is the unconditioned response. The increase in brightness 
of the light is Llled the conditioned stimulus, and the learned response of 
blinking to it is the conditioned response. The whole learning sequence is 

known as conditioning. 



APPENDIX C 



Material Irrelevant to the Principle to be 
Learned for Eliciting Irrelevant 



Pre- Reinforcement Activity 
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Irrelevant Material 
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German 

1. lachen 

2. reden 

3. erlau’ben 

4. leben 

5. schicken 

6. desto 

7. bei 

8. errei'chen 

9. an’-zichen 

10. weaken 

11 o begeg’nen 

12. drucken 

13. fressen 

14. Zwischen 

15. suchen 



English 

1 . to laugh 

2. to talk 

3. allow 

4. to praise 

5 . send 

6. the 

7. with 

8 . reach 

9 . put on 

10 . wake up 

11. meet 

12. press 

13. devour 

14. between 

15. seek 






APPENDIX D 

Criterion for Evaluating the Essay- 
Questions of Test Part I 



Criterion for Evaluating Test Part I 
Criteria 

Principle is correctly stated or substantially correct 
(r'low some incompleteness or vagueness). 

Approximate principle, but relationship between variables 
not correct, or some variables left out. 

Correct example only. 

Wrong example, wrong principle, failure to answer. 

Note: combination of answers do not get additional 

credit. Thus, approximate principle and a correct 
example receive 1 point only. 
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BACKGROUND 



The view that learning varies inversely with the amount of time inter- 
vening between a response and reinforcement is incorporated in several 
prominent theoretical approaches to learning (Hull. 1952; Spence. 1956; 
Miller. 1950; Skinner. 1938. 1957). Furthermore, this notion of the 
superiority of immediate reinforcement has been included in almost all 
attempts to apply principles of learning to education practices. 

However, evidence for the superiority of immediate reinforcement for 
cognitive learning is inconducive. It is questionable whether generalities 
based primarily upon motor responses of other experiences are applicable 

to the learning of typical academic material. 

This study explored the effect of delayed feedbac n principle 

learning. 

OBJECTIVES 

1. To compare the effect of immediate versus delayed feedback. 

2. To explore the interaction of immediate and delayed feedback upon 

deductive and inductive modes of learning. 

3. To explore the effect of the presence or absence of material 
supporting relevant symbolic activity as it interacts with the 



above verbals. 



2 



PROCEDURE 

A 2x2x3 factorial design was utilized to explore the effects and interactions 

between the three variables of concern; 

Variable A; The temporal interval between response and reinforcement. 
Variable B: The presence of non- informative material designed to pro- 

mote relevant symbolic activity during the pre-reinforcement 

interval . 

Variable C: The method of learnings -inductive- or deductive. 

The subjects were two hundred and seventy-seven (277) lower division 

college students. 

The learning materials used in the experiment consisted of six verbally 
expressed principles and exemplars and' non- exemplars of each principle such 

as: 

1. Subjects prior to the treatment would be unable to identify principles 
or examples at greater than chance levels. 

2. Subjects could displaysome evidence of learning after relatively 

brief treatments. 

3. Principles were similar to those found in lower division college course 
materials . 

4. A number of exemplars and non-exemplars of each principle could be 
identified. 

Two by two by three factor analysis of variance, using the hypergeometric 
correction, is used to analyse the data. Data thus analyzed consisted of correct 
matching of principles with new exemplars. 
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RESULTS AND CONCLUSIONS 

Only one main effect was significant: The kind of interpolated material 

presented. Material designed to promote relevant symbolic activity resulted 
in less learning than either material designed to promote non-relevant sym- 
bolic activity or no material. One interaction was significant, that between 
mode of instruction and temporal interval of feedback. Whether immediate or 
delayed feedback was more advisable, depended on whether the mode of learning 
was inductive or deductive. Deductive learning was best supported by delayed 
knowledge of results; inductive learning by immediate knowledge of results. 
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